Skip to content

ci(repo): add e2e testing infrastructure (iOS + Android)#2777

Merged
testableapple merged 13 commits into
masterfrom
ci/e2e-testing-infra
Jul 16, 2026
Merged

ci(repo): add e2e testing infrastructure (iOS + Android)#2777
testableapple merged 13 commits into
masterfrom
ci/e2e-testing-infra

Conversation

@testableapple

@testableapple testableapple commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator

Resolve https://linear.app/stream/issue/FLU-523

What

Adds end-to-end testing for sample_app on iOS and Android using Flutter's native integration_test runner (flutter test integration_test/ -d <device>), against the shared stream-chat-test-mock-server.

Mirrors the Robot / PageObject / BDD conventions from the native Android & iOS SDKs.

How it's wired

  • Mock server integration: tests run in-process, so the mock-server URL is injected via debugConnectionOverride on the AuthController singleton. isE2eTestRun guards Firebase/push/notifications during boot, SDK logging is disabled for e2e, and AuthController.debugReset() isolates state between tests.
  • Robots & DSL: MockServer, BackendRobot, ParticipantRobot (ports of the Android robots), fluent UserRobot + assertion extensions, page objects, and StreamTestEnv + step() BDD wrapper.
  • Selectors: reuse identifiers the SDK already exposes (widget types / existing keys) — no test-only keys added to the SDK.
  • Allure: Dart reporter (integration_test/allure/) emits Allure 2 results via chunked log markers; Allurefile reassembles them from device logs (adb logcat / simctl log) and uploads to Allure TestOps (project 135).
  • Fastlane: shared Fastfile (mock-server + e2e lanes), Allurefile, and Logfile (whitelist console output in CI; full unfiltered log written to build/e2e-test.log).
  • CI: e2e_test.yml (PR + manual dispatch, Android + iOS) and nightly e2e_test_cron.yml (API-level / iOS runtime matrix + Slack on failure). Full e2e log uploaded as a workflow artifact on failure.

Test

Summary by CodeRabbit

Summary

  • New Features

    • Added richer integration end-to-end coverage for messaging flows (lists, replies, reactions, attachments, and GIF interactions) with step-based reporting.
    • Introduced a mock backend–driven E2E mode with automated result reporting via Allure TestOps.
    • Added automated CI E2E runs for Android and iOS, plus a new nightly scheduled workflow.
  • Bug Fixes

    • Improved E2E reliability by isolating app network/log behavior and ensuring clean teardown.
  • Chores

    • Improved CI speed and consistency with new caching actions, consolidated setup steps, and better artifact/log handling.

@coderabbitai

coderabbitai Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds a Flutter integration-test harness with mock-server control, Allure reporting, Fastlane orchestration, production connection overrides, reusable CI actions, platform setup, and pull-request/nightly Android and iOS workflows.

Changes

Flutter E2E Testing

Layer / File(s) Summary
Reusable CI setup
.github/actions/*, .github/workflows/distribute_*.yml, .github/workflows/stream_flutter_workflow.yml
Centralizes Flutter, Ruby, Java, caching, KVM, and iOS runtime setup through local composite actions.
Production test seam
sample_app/lib/auth/auth_controller.dart, sample_app/lib/app.dart, sample_app/ios/Runner/Info.plist
Routes E2E connections to mock HTTP/WebSocket endpoints and skips production-only notification, push-token, and logging behavior.
Dart test harness and robots
sample_app/integration_test/*
Adds Allure serialization, mock-server control, test environment lifecycle, widget helpers, page abstractions, backend/participant/user robots, and a message-list integration test.
Fastlane and platform orchestration
sample_app/Fastfile, sample_app/fastlane/*, sample_app/ios/fastlane/*, sample_app/scripts/*
Adds mock-server startup, retryable E2E execution, Allure launch/result handling, iOS runtime installation, and platform-specific Fastlane wiring.
E2E workflows and project configuration
.github/workflows/e2e_test*.yml, melos.yaml, sample_app/pubspec.yaml, sample_app/android/*, .gitignore
Adds on-demand and scheduled matrix workflows, E2E dependencies and scripts, Android build updates, Ruby configuration, and generated-artifact ignores.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Workflow as GitHub Actions workflow
  participant Fastlane as Fastlane E2E lane
  participant MockServer as MockServer
  participant App as Flutter sample app
  participant Allure as Allure TestOps

  Workflow->>Fastlane: run_e2e_test
  Fastlane->>MockServer: start driver and backend
  Fastlane->>App: run integration test
  App->>MockServer: route HTTP/WebSocket traffic
  App->>Allure: emit encoded test results
  Fastlane->>Allure: collect and upload results
  Fastlane->>MockServer: stop driver
Loading

Suggested reviewers: renefloor, velikovpetar

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: adding iOS and Android end-to-end testing infrastructure.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ci/e2e-testing-infra

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@testableapple
testableapple marked this pull request as draft June 22, 2026 16:26
Comment thread .github/workflows/e2e_test.yml Fixed
Comment thread .github/workflows/e2e_test.yml Fixed
Comment thread .github/workflows/e2e_test_cron.yml Fixed
Comment thread .github/workflows/e2e_test_cron.yml Fixed
Comment thread .github/workflows/e2e_test_cron.yml Fixed
@testableapple testableapple changed the title test: add Patrol-based e2e testing infrastructure (iOS + Android) [WIP] test: add e2e testing infrastructure (iOS + Android) Jun 22, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 10

Note

Due to the large number of review comments, Critical, Major severity comments were prioritized as inline comments.

🟡 Minor comments (13)
.github/workflows/e2e_test.yml-108-111 (1)

108-111: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Add timeout-minutes to the iOS e2e step.

The iOS test run lacks an explicit timeout, similar to the Android job.

🔧 Proposed fix
       - name: Run e2e (iOS simulator)
+        timeout-minutes: 60
         run: |
           cd sample_app/ios
           bundle exec fastlane run_e2e_test device:${{ env.device_id }} mock_server_branch:main
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/e2e_test.yml around lines 108 - 111, The "Run e2e (iOS
simulator)" step in the iOS e2e workflow is missing an explicit timeout
configuration. Add a `timeout-minutes` property to this step to set a maximum
execution time for the iOS simulator test run, similar to what exists in the
Android job to prevent the workflow from running indefinitely in case of
failures or hangs.
.github/workflows/e2e_test_cron.yml-135-136 (1)

135-136: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Add timeout-minutes to the iOS e2e step for consistency with Android.

The Android job has timeout-minutes: 90 on the emulator runner, but the iOS job lacks an explicit timeout.

🔧 Proposed fix
       - name: Run e2e (iOS simulator)
+        timeout-minutes: 90
         run: cd sample_app/ios && bundle exec fastlane run_e2e_test device:${{ env.device_id }} mock_server_branch:${{ env.mock_server_branch }}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/e2e_test_cron.yml around lines 135 - 136, The iOS e2e test
step named "Run e2e (iOS simulator)" is missing the timeout-minutes
configuration that the Android job has. Add a timeout-minutes field set to 90 to
this step to maintain consistency with the Android emulator runner's timeout
configuration. This should be added as a step-level property at the same
indentation level as the name and run fields.
.github/workflows/e2e_test.yml-53-61 (1)

53-61: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Add timeout-minutes to the Android emulator step.

Unlike the nightly workflow which specifies timeout-minutes: 90, this step lacks a timeout. If tests hang, the job will run until the workflow's default timeout (6 hours), wasting compute and blocking the concurrency group.

🔧 Proposed fix
       - name: Run e2e (Android emulator)
         uses: reactivecircus/android-emulator-runner@v2
+        timeout-minutes: 60
         with:
           api-level: 34
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/e2e_test.yml around lines 53 - 61, The "Run e2e (Android
emulator)" step that uses reactivecircus/android-emulator-runner@v2 is missing a
timeout-minutes configuration. Add timeout-minutes: 90 as a property at the same
indentation level as the uses and with properties to prevent tests from hanging
indefinitely and consuming resources until the default 6-hour workflow timeout
is reached.
sample_app/integration_test/pages/message_list_page.dart-4-6 (1)

4-6: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Add doc comments for MessageListPage public API.

MessageListPage (and exposed composer) is public and should be documented.

As per coding guidelines: “All public APIs must have doc comments (public_member_api_docs)”.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sample_app/integration_test/pages/message_list_page.dart` around lines 4 - 6,
The MessageListPage class and its public static constant composer are missing
documentation comments required by the coding guidelines. Add doc comments
(starting with ///) to both the MessageListPage abstract final class and the
static const composer field to document their purpose and usage, ensuring all
public APIs are properly documented.

Source: Coding guidelines

sample_app/integration_test/mock_server/data_types.dart-1-25 (1)

1-25: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Add doc comments for exported enums and constants.

AttachmentType, ReactionType, MessageDeliveryStatus, and forbiddenWord are public APIs and need Dart doc comments.

As per coding guidelines: “All public APIs must have doc comments (public_member_api_docs)”.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sample_app/integration_test/mock_server/data_types.dart` around lines 1 - 25,
Add Dart doc comments to the public APIs in this file to comply with the
public_member_api_docs guideline. For the three enums AttachmentType,
ReactionType, and MessageDeliveryStatus, add doc comments above their enum
declarations that describe their purpose and usage. Similarly, add a doc comment
above the forbiddenWord constant that explains what it represents. Use the
standard Dart doc comment format (///) for all public members to ensure they are
properly documented.

Source: Coding guidelines

sample_app/integration_test/robots/backend_robot.dart-1-54 (1)

1-54: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Apply package imports and doc comments for public robot API.

This file uses a relative import and exposes public APIs without Dart doc comments.

As per coding guidelines: “Always use package imports instead of relative imports (always_use_package_imports)” and “All public APIs must have doc comments (public_member_api_docs)”.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sample_app/integration_test/robots/backend_robot.dart` around lines 1 - 54,
Replace the relative import statement with a package import for the mock_server
module in the BackendRobot file. Additionally, add Dart doc comments (using ///)
to all public APIs including the BackendRobot class definition and all public
methods: generateChannels, failNewMessages, freezeNewMessages, revokeToken,
invalidateToken, invalidateTokenDate, invalidateTokenSignature, and
breakTokenGeneration. Each doc comment should clearly describe the purpose and
behavior of the method or class.

Source: Coding guidelines

sample_app/integration_test/mock_server/mock_server.dart-7-87 (1)

7-87: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Document the public MockServer API.

MockServer and its public members are exported test infrastructure APIs and should have Dart doc comments for maintainability.

As per coding guidelines: “All public APIs must have doc comments (public_member_api_docs)”.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sample_app/integration_test/mock_server/mock_server.dart` around lines 7 -
87, Add Dart doc comments to all public members of the MockServer class to
comply with the public_member_api_docs guideline. Document the MockServer class
itself, and all public methods and properties: the url and wsUrl fields, the
start() static factory method, and the instance methods stop(), post(), get(),
and waitUntilReady(). Each comment should explain the purpose and usage of the
respective public API member.

Source: Coding guidelines

sample_app/integration_test/robots/participant_robot.dart-1-176 (1)

1-176: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Switch to package imports and document the public robot surface.

This file uses relative imports and exports a large public API (ParticipantRobot methods) without doc comments.

As per coding guidelines: “Always use package imports instead of relative imports (always_use_package_imports)” and “All public APIs must have doc comments (public_member_api_docs)”.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sample_app/integration_test/robots/participant_robot.dart` around lines 1 -
176, The ParticipantRobot class in participant_robot.dart has two issues: the
imports at the top use relative paths instead of package imports, and all public
methods lack documentation comments. Replace the relative imports (using ../
paths) with package imports using the package: prefix for the data_types and
mock_server modules. Then add doc comments to the ParticipantRobot class and all
its public methods including startTyping, stopTyping, sendMessage, editMessage,
deleteMessage, quoteMessage, uploadGiphy, pinMessage, unpinMessage,
uploadAttachment, addReaction, deleteReaction, and their related variants to
document their purpose and behavior.

Source: Coding guidelines

sample_app/integration_test/support/stream_test_env.dart-5-8 (1)

5-8: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Use package imports and add doc comments for public test env API.

This file uses relative imports and exposes public APIs (StreamTestEnv, setUp, tearDown) without doc comments.

As per coding guidelines: “Always use package imports instead of relative imports (always_use_package_imports)” and “All public APIs must have doc comments (public_member_api_docs)”.

Also applies to: 10-34

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sample_app/integration_test/support/stream_test_env.dart` around lines 5 - 8,
Replace all relative imports in this file with package imports by converting
imports like '../mock_server/mock_server.dart', '../robots/backend_robot.dart',
'../robots/participant_robot.dart', and '../robots/user_robot.dart' to use the
package: prefix (e.g., 'package:sample_app/...'). Additionally, add
documentation comments to all public APIs exposed by this file, specifically the
StreamTestEnv class, setUp function, and tearDown function, ensuring each public
member has a clear doc comment explaining its purpose and usage.

Source: Coding guidelines

sample_app/integration_test/support/stream_test.dart-9-13 (1)

9-13: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Document the public streamTest wrapper.

streamTest is public and should have a doc comment describing intended usage and behavior.

As per coding guidelines: **/*.dart: All public APIs must have doc comments (public_member_api_docs).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sample_app/integration_test/support/stream_test.dart` around lines 9 - 13,
The public streamTest function lacks a doc comment which is required by the
public_member_api_docs guideline. Add a documentation comment above the
streamTest function that clearly describes its purpose as a test wrapper,
documents the parameters (description, callback, and allureId), and explains its
intended usage and behavior for stream-based integration testing.

Source: Coding guidelines

sample_app/integration_test/PLAN.md-233-237 (1)

233-237: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Specify languages for fenced code blocks.

Two fenced blocks are missing language identifiers, triggering MD040. Add a language like text, dart, kotlin, or ruby as appropriate.

Also applies to: 275-321

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sample_app/integration_test/PLAN.md` around lines 233 - 237, The fenced code
blocks in PLAN.md at lines 233-237 and 275-321 are missing language identifiers
after the opening triple backticks, which triggers the MD040 linting rule. Add
an appropriate language identifier (such as `text`, `ruby`, or `dart`)
immediately after the triple backticks for each fenced code block to specify the
content type and resolve the linting violation.

Source: Linters/SAST tools

sample_app/integration_test/allure/allure.dart-7-140 (1)

7-140: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Add doc comments for the exported Allure API.

This file exposes public members (allureResultMarker, AllureStatus, Allure, and public methods) without API docs, which will trip public_member_api_docs in linted runs.

As per coding guidelines: **/*.dart: All public APIs must have doc comments (public_member_api_docs).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sample_app/integration_test/allure/allure.dart` around lines 7 - 140, Add
documentation comments (/// style) to all public members in the file to satisfy
the public_member_api_docs lint rule. This includes the allureResultMarker
constant, the AllureStatus enum and its enum values, the Allure class, and its
public methods (startTest, step, and stopTest). Each doc comment should clearly
describe the purpose and usage of the member, ensuring that all exported APIs
have proper documentation.

Source: Coding guidelines

sample_app/integration_test/support/stream_test.dart-7-7 (1)

7-7: ⚠️ Potential issue | 🟡 Minor

Switch this relative import to a package import.

Use package:sample_app/integration_test/allure/allure.dart instead of the relative path. Relative imports make refactoring across directories more fragile and violate the always_use_package_imports guideline.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sample_app/integration_test/support/stream_test.dart` at line 7, Replace the
relative import statement in stream_test.dart with a package import to improve
maintainability and follow the always_use_package_imports guideline. Change the
import from using the relative path '../allure/allure.dart' to use the full
package path format 'package:sample_app/integration_test/allure/allure.dart'
instead. This makes the codebase more resilient to directory refactoring and
improves code clarity.

Source: Coding guidelines

🧹 Nitpick comments (7)
.github/workflows/e2e_test_cron.yml (1)

162-168: 🧹 Nitpick | 🔵 Trivial | ⚖️ Poor tradeoff

Consider replacing archived Slack action.

The 8398a7/action-slack@v3 repository is archived. While it still functions, consider migrating to an actively maintained alternative like slackapi/slack-github-action for long-term support.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/e2e_test_cron.yml around lines 162 - 168, Replace the
archived `8398a7/action-slack@v3` action with the actively maintained
`slackapi/slack-github-action`. Update the uses statement to reference the new
action, and adjust the input parameters accordingly since the new action has a
different API - the text and fields parameters should be replaced with the
appropriate payload structure expected by slackapi/slack-github-action, and
ensure the SLACK_WEBHOOK_URL environment variable is properly passed to the new
action's configuration.

Source: Linters/SAST tools

sample_app/integration_test/support/step.dart (1)

1-4: 🧹 Nitpick | 🔵 Trivial | ⚡ Quick win

Use package import and document the public helper API.

This helper is fine functionally, but it violates the Dart style rules configured for this repo (relative import + missing doc comment on a public API).

Suggested patch
-import '../allure/allure.dart';
+import 'package:sample_app/integration_test/allure/allure.dart';
 
+/// Runs an Allure step with the provided [description].
 Future<T> step<T>(String description, Future<T> Function() body) =>
     Allure.instance.step(description, body);

As per coding guidelines, **/*.dart must use package imports and all public APIs must have doc comments.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sample_app/integration_test/support/step.dart` around lines 1 - 4, The import
statement for the allure module uses a relative import path, but the repo
requires package imports for all Dart files. Additionally, the public step
function is missing a documentation comment as required by the Dart style
guidelines. Convert the relative import statement to use the package import
syntax, and add a doc comment above the step function definition that documents
its purpose, parameters, and return type to comply with the coding standards for
public APIs.

Source: Coding guidelines

sample_app/integration_test/pages/channel_list_page.dart (1)

3-5: 🧹 Nitpick | 🔵 Trivial | ⚡ Quick win

Document public page-object API members.

ChannelListPage and its public channelTile member should have doc comments to satisfy the repo Dart API documentation rule.

As per coding guidelines, all public APIs in **/*.dart must have doc comments.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sample_app/integration_test/pages/channel_list_page.dart` around lines 3 - 5,
Add doc comments to the public API members in the ChannelListPage class to
comply with the Dart documentation requirement. The ChannelListPage class itself
needs a doc comment explaining its purpose, and the static channelTile constant
member needs a doc comment describing what it represents. Place the doc comments
immediately before each member declaration using the triple-slash (///) syntax
to ensure the repository's API documentation generation includes these
definitions.

Source: Coding guidelines

sample_app/integration_test/robots/user_robot.dart (1)

3-33: 🧹 Nitpick | 🔵 Trivial | ⚡ Quick win

Replace relative imports and add docs for the public robot API.

The robot implementation looks good, but this file currently violates Dart repo rules on package imports and public API docs.

Suggested import direction
-import '../pages/channel_list_page.dart';
-import '../pages/message_list_page.dart';
-import '../support/predefined_users.dart';
+import 'package:sample_app/integration_test/pages/channel_list_page.dart';
+import 'package:sample_app/integration_test/pages/message_list_page.dart';
+import 'package:sample_app/integration_test/support/predefined_users.dart';

As per coding guidelines, **/*.dart must use package imports and all public APIs must have doc comments.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sample_app/integration_test/robots/user_robot.dart` around lines 3 - 33,
Replace all relative imports in the UserRobot class file with absolute package
imports (convert imports like '../pages/channel_list_page.dart' to use the
package: syntax). Additionally, add documentation comments using the /// syntax
to the public UserRobot class and all its public methods including login,
openChannel, and sendMessage to document their purpose, parameters, and return
values according to Dart documentation guidelines.

Source: Coding guidelines

sample_app/integration_test/support/predefined_users.dart (1)

3-26: 🧹 Nitpick | 🔵 Trivial | ⚡ Quick win

Add public API docs and split the token literal to satisfy max line length.

This file exposes public APIs without doc comments, and the JWT line likely exceeds the 120-char limit. Please document UserCredentials/PredefinedUsers and break long literals across adjacent string segments.

As per coding guidelines, public APIs in **/*.dart require doc comments and lines in **/*.{dart,dart.bak} must not exceed 120 characters.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sample_app/integration_test/support/predefined_users.dart` around lines 3 -
26, Add doc comments to the public classes UserCredentials and PredefinedUsers
to document their purpose and usage, and split the long JWT token literal in the
qaTest1 constant across multiple adjacent string segments to ensure no line
exceeds the 120 character limit as per coding guidelines.

Source: Coding guidelines

sample_app/integration_test/robots/user_robot_message_list_asserts.dart (1)

1-7: 🧹 Nitpick | 🔵 Trivial | ⚡ Quick win

Make the extension file lint-compliant (package import + API docs).

Please switch to package import and add doc comments for the public extension and assertion method.

As per coding guidelines, Dart files must use package imports and public APIs must have doc comments.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sample_app/integration_test/robots/user_robot_message_list_asserts.dart`
around lines 1 - 7, The file uses a relative import statement instead of a
package import, and the public extension UserRobotMessageListAsserts and its
public method assertMessage lack doc comments. Replace the relative import with
a package import statement that includes the appropriate package path, and add
doc comments (using ///) to both the UserRobotMessageListAsserts extension and
the assertMessage method to document their purpose and usage.

Source: Coding guidelines

sample_app/integration_test/message_list_test.dart (1)

3-6: 🧹 Nitpick | 🔵 Trivial | ⚡ Quick win

Use package imports in the integration test file.

Please replace these relative imports with package:sample_app/... imports to satisfy the Dart import policy.

As per coding guidelines, **/*.dart should always use package imports instead of relative imports.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sample_app/integration_test/message_list_test.dart` around lines 3 - 6,
Replace all four relative imports at the top of the message_list_test.dart file
with package imports. Convert the imports of
user_robot_message_list_asserts.dart, step.dart, stream_test.dart, and
stream_test_env.dart from relative paths (using ./) to package imports using the
format package:sample_app/path/to/file.dart. Ensure the paths after
package:sample_app/ reflect the correct location of these imported files within
the project structure.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/e2e_test_cron.yml:
- Around line 62-65: The current workflow creates a separate Allure launch for
each matrix cell (3 Android variants and 2 iOS variants), but they should all
upload to a single shared launch. Create a new `setup` job that runs once before
the matrix jobs to generate a launch_id, then extract that launch_id as an
output so the android and ios jobs can consume it via `needs: setup`. Remove the
"Create Allure launch" step from both the android and ios job definitions, and
update their "Upload Allure results" steps to use the shared launch_id from the
setup job output instead of creating new launches in each matrix cell.
- Around line 1-20: The E2E Tests Nightly workflow in the e2e_test_cron.yml file
is missing explicit minimal permissions at the workflow level. Add a
workflow-level permissions block to the file (after the concurrency section or
at the top level, similar to how other workflow files are structured) and define
explicit minimal permissions that match what is configured in the corresponding
on-demand workflow. This ensures the workflow operates with the principle of
least privilege.

In @.github/workflows/e2e_test.yml:
- Around line 1-16: Add a workflow-level permissions block to the e2e workflow
to restrict permissions to only what is needed. The permissions block should be
added after the 'on:' section (after the pull_request trigger configuration) and
before the 'concurrency:' section. Since this workflow only needs to checkout
code and upload artifacts, add a permissions block that specifies contents: read
for code checkout, and any other minimal permissions required for the specific
operations the workflow performs. This replaces the default overly-broad
permissions with an explicitly minimal set.

In `@sample_app/Allurefile`:
- Around line 14-18: The adb logcat command in the Android branch does not
utilize the options[:device] parameter, which causes it to retrieve logs from
the default device when multiple devices are connected. Modify the adb logcat -d
command to include the device specifier by passing options[:device] to the adb
command using the -s flag (adb -s <device> syntax) so that it respects the
device parameter consistently with the iOS branch that already uses
options[:device] for simctl.

In `@sample_app/Fastfile`:
- Line 59: The adb logcat clear command in the current implementation does not
specify the target device when options[:device] is provided. Modify the sh('adb
logcat -c') command to include the device specification flag (such as -s with
the device name) when options[:device] is present, making it consistent with the
device targeting pattern used in the collect_allure_results function. Ensure the
device parameter is only appended to the command when it is available in the
options hash.

In `@sample_app/integration_test/mock_server/mock_server.dart`:
- Around line 18-24: The MockServer.start() method makes an HTTP call via _get()
to the mock driver without an explicit timeout, which can cause indefinite hangs
if the driver is unresponsive. Add explicit request timeouts to the _get(),
_statusCode(), and post() helper methods to prevent blocking indefinitely. Each
HTTP request should include a timeout duration parameter (typically in the range
of seconds) to ensure the application can fail fast and proceed to error
handling or retry logic when the driver is unresponsive.

In `@sample_app/integration_test/robots/backend_robot.dart`:
- Around line 15-26: The messagesText and repliesText parameters are being
directly concatenated into the query string without URL-encoding, which will
cause the request parameters to be corrupted if these values contain reserved
characters like ampersands, question marks, spaces, or equals signs. Apply
proper URL-encoding to the messagesText and repliesText values before they are
inserted into the messagesTextParam and repliesTextParam string construction.
Use Dart's URI encoding utility to ensure these parameters are safely encoded
for the query string in the _mockServer.post() call.

In `@sample_app/integration_test/support/predefined_users.dart`:
- Around line 18-23: Remove the hardcoded token from the `qaTest1`
UserCredentials constant in the `PredefinedUsers` class. Instead of embedding
the JWT directly in the fixture, implement a mechanism to generate short-lived
test tokens dynamically from the mock server on each test run. Alternatively,
inject the token from secure CI secrets at runtime. Update the `UserCredentials`
class or the test initialization to accept dynamically generated tokens rather
than relying on static fixture values.

In `@sample_app/integration_test/support/stream_test_env.dart`:
- Around line 30-33: The tearDown() method in the stream test environment class
does not handle exceptions properly - if authController.debugReset() throws an
error, the mockServer.stop() call will be skipped, leaving the server in an
unclean state for subsequent tests. Wrap the teardown logic in a try/finally
block where authController.debugReset() is in the try block and
mockServer.stop() is in the finally block to ensure the mock server is always
stopped regardless of whether the auth controller reset succeeds or fails.

In `@sample_app/ios/Runner.xcodeproj/project.pbxproj`:
- Line 83: Replace the hardcoded SDK version path in the Foundation.framework
file reference (A6B213AE2D543B7BADA38E33). Instead of using the explicit path
containing `iPhoneOS18.0.sdk`, update the path attribute to use a dynamic
SDK-relative reference such as using the `SDKROOT` variable or a relative path
pattern like `System/Library/Frameworks/Foundation.framework` that will work
across different Xcode versions and CI environments. Apply the same fix to other
SDK version-pinned framework references mentioned at lines 103-105.

---

Minor comments:
In @.github/workflows/e2e_test_cron.yml:
- Around line 135-136: The iOS e2e test step named "Run e2e (iOS simulator)" is
missing the timeout-minutes configuration that the Android job has. Add a
timeout-minutes field set to 90 to this step to maintain consistency with the
Android emulator runner's timeout configuration. This should be added as a
step-level property at the same indentation level as the name and run fields.

In @.github/workflows/e2e_test.yml:
- Around line 108-111: The "Run e2e (iOS simulator)" step in the iOS e2e
workflow is missing an explicit timeout configuration. Add a `timeout-minutes`
property to this step to set a maximum execution time for the iOS simulator test
run, similar to what exists in the Android job to prevent the workflow from
running indefinitely in case of failures or hangs.
- Around line 53-61: The "Run e2e (Android emulator)" step that uses
reactivecircus/android-emulator-runner@v2 is missing a timeout-minutes
configuration. Add timeout-minutes: 90 as a property at the same indentation
level as the uses and with properties to prevent tests from hanging indefinitely
and consuming resources until the default 6-hour workflow timeout is reached.

In `@sample_app/integration_test/allure/allure.dart`:
- Around line 7-140: Add documentation comments (/// style) to all public
members in the file to satisfy the public_member_api_docs lint rule. This
includes the allureResultMarker constant, the AllureStatus enum and its enum
values, the Allure class, and its public methods (startTest, step, and
stopTest). Each doc comment should clearly describe the purpose and usage of the
member, ensuring that all exported APIs have proper documentation.

In `@sample_app/integration_test/mock_server/data_types.dart`:
- Around line 1-25: Add Dart doc comments to the public APIs in this file to
comply with the public_member_api_docs guideline. For the three enums
AttachmentType, ReactionType, and MessageDeliveryStatus, add doc comments above
their enum declarations that describe their purpose and usage. Similarly, add a
doc comment above the forbiddenWord constant that explains what it represents.
Use the standard Dart doc comment format (///) for all public members to ensure
they are properly documented.

In `@sample_app/integration_test/mock_server/mock_server.dart`:
- Around line 7-87: Add Dart doc comments to all public members of the
MockServer class to comply with the public_member_api_docs guideline. Document
the MockServer class itself, and all public methods and properties: the url and
wsUrl fields, the start() static factory method, and the instance methods
stop(), post(), get(), and waitUntilReady(). Each comment should explain the
purpose and usage of the respective public API member.

In `@sample_app/integration_test/pages/message_list_page.dart`:
- Around line 4-6: The MessageListPage class and its public static constant
composer are missing documentation comments required by the coding guidelines.
Add doc comments (starting with ///) to both the MessageListPage abstract final
class and the static const composer field to document their purpose and usage,
ensuring all public APIs are properly documented.

In `@sample_app/integration_test/PLAN.md`:
- Around line 233-237: The fenced code blocks in PLAN.md at lines 233-237 and
275-321 are missing language identifiers after the opening triple backticks,
which triggers the MD040 linting rule. Add an appropriate language identifier
(such as `text`, `ruby`, or `dart`) immediately after the triple backticks for
each fenced code block to specify the content type and resolve the linting
violation.

In `@sample_app/integration_test/robots/backend_robot.dart`:
- Around line 1-54: Replace the relative import statement with a package import
for the mock_server module in the BackendRobot file. Additionally, add Dart doc
comments (using ///) to all public APIs including the BackendRobot class
definition and all public methods: generateChannels, failNewMessages,
freezeNewMessages, revokeToken, invalidateToken, invalidateTokenDate,
invalidateTokenSignature, and breakTokenGeneration. Each doc comment should
clearly describe the purpose and behavior of the method or class.

In `@sample_app/integration_test/robots/participant_robot.dart`:
- Around line 1-176: The ParticipantRobot class in participant_robot.dart has
two issues: the imports at the top use relative paths instead of package
imports, and all public methods lack documentation comments. Replace the
relative imports (using ../ paths) with package imports using the package:
prefix for the data_types and mock_server modules. Then add doc comments to the
ParticipantRobot class and all its public methods including startTyping,
stopTyping, sendMessage, editMessage, deleteMessage, quoteMessage, uploadGiphy,
pinMessage, unpinMessage, uploadAttachment, addReaction, deleteReaction, and
their related variants to document their purpose and behavior.

In `@sample_app/integration_test/support/stream_test_env.dart`:
- Around line 5-8: Replace all relative imports in this file with package
imports by converting imports like '../mock_server/mock_server.dart',
'../robots/backend_robot.dart', '../robots/participant_robot.dart', and
'../robots/user_robot.dart' to use the package: prefix (e.g.,
'package:sample_app/...'). Additionally, add documentation comments to all
public APIs exposed by this file, specifically the StreamTestEnv class, setUp
function, and tearDown function, ensuring each public member has a clear doc
comment explaining its purpose and usage.

In `@sample_app/integration_test/support/stream_test.dart`:
- Around line 9-13: The public streamTest function lacks a doc comment which is
required by the public_member_api_docs guideline. Add a documentation comment
above the streamTest function that clearly describes its purpose as a test
wrapper, documents the parameters (description, callback, and allureId), and
explains its intended usage and behavior for stream-based integration testing.
- Line 7: Replace the relative import statement in stream_test.dart with a
package import to improve maintainability and follow the
always_use_package_imports guideline. Change the import from using the relative
path '../allure/allure.dart' to use the full package path format
'package:sample_app/integration_test/allure/allure.dart' instead. This makes the
codebase more resilient to directory refactoring and improves code clarity.

---

Nitpick comments:
In @.github/workflows/e2e_test_cron.yml:
- Around line 162-168: Replace the archived `8398a7/action-slack@v3` action with
the actively maintained `slackapi/slack-github-action`. Update the uses
statement to reference the new action, and adjust the input parameters
accordingly since the new action has a different API - the text and fields
parameters should be replaced with the appropriate payload structure expected by
slackapi/slack-github-action, and ensure the SLACK_WEBHOOK_URL environment
variable is properly passed to the new action's configuration.

In `@sample_app/integration_test/message_list_test.dart`:
- Around line 3-6: Replace all four relative imports at the top of the
message_list_test.dart file with package imports. Convert the imports of
user_robot_message_list_asserts.dart, step.dart, stream_test.dart, and
stream_test_env.dart from relative paths (using ./) to package imports using the
format package:sample_app/path/to/file.dart. Ensure the paths after
package:sample_app/ reflect the correct location of these imported files within
the project structure.

In `@sample_app/integration_test/pages/channel_list_page.dart`:
- Around line 3-5: Add doc comments to the public API members in the
ChannelListPage class to comply with the Dart documentation requirement. The
ChannelListPage class itself needs a doc comment explaining its purpose, and the
static channelTile constant member needs a doc comment describing what it
represents. Place the doc comments immediately before each member declaration
using the triple-slash (///) syntax to ensure the repository's API documentation
generation includes these definitions.

In `@sample_app/integration_test/robots/user_robot_message_list_asserts.dart`:
- Around line 1-7: The file uses a relative import statement instead of a
package import, and the public extension UserRobotMessageListAsserts and its
public method assertMessage lack doc comments. Replace the relative import with
a package import statement that includes the appropriate package path, and add
doc comments (using ///) to both the UserRobotMessageListAsserts extension and
the assertMessage method to document their purpose and usage.

In `@sample_app/integration_test/robots/user_robot.dart`:
- Around line 3-33: Replace all relative imports in the UserRobot class file
with absolute package imports (convert imports like
'../pages/channel_list_page.dart' to use the package: syntax). Additionally, add
documentation comments using the /// syntax to the public UserRobot class and
all its public methods including login, openChannel, and sendMessage to document
their purpose, parameters, and return values according to Dart documentation
guidelines.

In `@sample_app/integration_test/support/predefined_users.dart`:
- Around line 3-26: Add doc comments to the public classes UserCredentials and
PredefinedUsers to document their purpose and usage, and split the long JWT
token literal in the qaTest1 constant across multiple adjacent string segments
to ensure no line exceeds the 120 character limit as per coding guidelines.

In `@sample_app/integration_test/support/step.dart`:
- Around line 1-4: The import statement for the allure module uses a relative
import path, but the repo requires package imports for all Dart files.
Additionally, the public step function is missing a documentation comment as
required by the Dart style guidelines. Convert the relative import statement to
use the package import syntax, and add a doc comment above the step function
definition that documents its purpose, parameters, and return type to comply
with the coding standards for public APIs.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ea4abfd1-bfdb-44c3-a296-04aa453ec6b9

📥 Commits

Reviewing files that changed from the base of the PR and between 6ab13f2 and dc20586.

⛔ Files ignored due to path filters (2)
  • sample_app/android/Gemfile.lock is excluded by !**/*.lock
  • sample_app/ios/Gemfile.lock is excluded by !**/*.lock
📒 Files selected for processing (33)
  • .github/workflows/e2e_test.yml
  • .github/workflows/e2e_test_cron.yml
  • .gitignore
  • melos.yaml
  • sample_app/Allurefile
  • sample_app/Fastfile
  • sample_app/android/app/build.gradle
  • sample_app/android/app/src/androidTest/java/io/getstream/chat/android/flutter/sample/MainActivityTest.java
  • sample_app/integration_test/PLAN.md
  • sample_app/integration_test/allure/allure.dart
  • sample_app/integration_test/message_list_test.dart
  • sample_app/integration_test/mock_server/data_types.dart
  • sample_app/integration_test/mock_server/mock_server.dart
  • sample_app/integration_test/pages/channel_list_page.dart
  • sample_app/integration_test/pages/message_list_page.dart
  • sample_app/integration_test/robots/backend_robot.dart
  • sample_app/integration_test/robots/participant_robot.dart
  • sample_app/integration_test/robots/user_robot.dart
  • sample_app/integration_test/robots/user_robot_message_list_asserts.dart
  • sample_app/integration_test/support/predefined_users.dart
  • sample_app/integration_test/support/step.dart
  • sample_app/integration_test/support/stream_test.dart
  • sample_app/integration_test/support/stream_test_env.dart
  • sample_app/ios/Podfile
  • sample_app/ios/Runner.xcodeproj/project.pbxproj
  • sample_app/ios/Runner.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved
  • sample_app/ios/Runner.xcodeproj/xcshareddata/xcschemes/Runner.xcscheme
  • sample_app/ios/Runner.xcworkspace/xcshareddata/swiftpm/Package.resolved
  • sample_app/ios/RunnerUITests/RunnerUITests.m
  • sample_app/ios/add_patrol_target.rb
  • sample_app/lib/app.dart
  • sample_app/lib/auth/auth_controller.dart
  • sample_app/pubspec.yaml

Comment thread .github/workflows/e2e_test_cron.yml
Comment thread .github/workflows/e2e_test_cron.yml Outdated
Comment thread .github/workflows/e2e_test.yml
Comment thread sample_app/Allurefile Outdated
Comment thread sample_app/Fastfile Outdated
Comment thread sample_app/integration_test/mock_server/mock_server.dart
Comment thread sample_app/integration_test/robots/backend_robot.dart Outdated
Comment thread sample_app/integration_test/support/predefined_users.dart Outdated
Comment thread sample_app/integration_test/support/stream_test_env.dart
Comment thread sample_app/ios/Runner.xcodeproj/project.pbxproj Outdated
Comment thread melos.yaml Outdated
@testableapple
testableapple force-pushed the ci/e2e-testing-infra branch 3 times, most recently from 24bfcf9 to 6bb1ce4 Compare July 9, 2026 22:34
@testableapple testableapple changed the title [WIP] test: add e2e testing infrastructure (iOS + Android) test: add e2e testing infrastructure (iOS + Android) Jul 10, 2026
@testableapple testableapple changed the title test: add e2e testing infrastructure (iOS + Android) ci: add e2e testing infrastructure (iOS + Android) Jul 10, 2026
@testableapple testableapple changed the title ci: add e2e testing infrastructure (iOS + Android) ci(repo): add e2e testing infrastructure (iOS + Android) Jul 10, 2026
@testableapple
testableapple force-pushed the ci/e2e-testing-infra branch 5 times, most recently from d5d47a3 to e8eec0f Compare July 15, 2026 08:11
@testableapple
testableapple force-pushed the ci/e2e-testing-infra branch from e8eec0f to c41ce39 Compare July 15, 2026 09:47
@testableapple
testableapple force-pushed the ci/e2e-testing-infra branch from 406beac to 406fcad Compare July 15, 2026 12:49
@testableapple
testableapple marked this pull request as ready for review July 15, 2026 12:49
@codecov

codecov Bot commented Jul 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.28%. Comparing base (e8e0466) to head (7c06473).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #2777   +/-   ##
=======================================
  Coverage   71.28%   71.28%           
=======================================
  Files         430      430           
  Lines       26934    26934           
=======================================
  Hits        19199    19199           
  Misses       7735     7735           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 9

🧹 Nitpick comments (6)
sample_app/integration_test/support/widget_test_extensions.dart (1)

14-18: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Prevent crashes on duplicate text matches.

If the provided text appears more than once on the screen (e.g., in a header and a button), find.text(text) will match multiple widgets, causing tap() to throw a StateError ("Too many elements").

Consider adding an optional index parameter (similar to how tapByType handles it) to make this helper resilient against duplicate text.

🛠 Proposed fix
-  Future<void> tapText(String text) async {
-    await waitUntilVisible(find.text(text));
-    await tap(find.text(text));
+  Future<void> tapText(String text, {int index = 0}) async {
+    final finder = find.text(text);
+    await waitUntilVisible(finder.at(index));
+    await tap(finder.at(index));
     await pumpAndSettle();
   }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sample_app/integration_test/support/widget_test_extensions.dart` around lines
14 - 18, Update tapText to accept an optional index parameter, resolve the text
finder to the indexed match when provided, and pass that single-widget finder to
tap so duplicate text matches do not cause a StateError; preserve the existing
behavior when no index is supplied.
.github/actions/setup-ios-runtime/action.yml (1)

34-39: 🔒 Security & Privacy | 🔵 Trivial | 💤 Low value

Prevent potential shell injection by mapping inputs to environment variables.

While the current inputs are controlled via matrix definitions, directly interpolating ${{ inputs.* }} into inline bash scripts can lead to syntax errors or shell injection if the inputs contain spaces, quotes, or special characters. It is safer to pass them as environment variables.

♻️ Proposed refactor
     - name: Create custom iOS simulator
       shell: bash
+      env:
+        IOS_VERSION: ${{ inputs.version }}
+        DEVICE_TYPE: ${{ inputs.device }}
       run: |
-        ios_version_dash=$(echo "${{ inputs.version }}" | tr '.' '-')
-        xcrun simctl create custom-test-device "${{ inputs.device }}" "com.apple.CoreSimulator.SimRuntime.iOS-${ios_version_dash}"
-        xcrun simctl list devices "${{ inputs.version }}"
+        ios_version_dash=$(echo "$IOS_VERSION" | tr '.' '-')
+        xcrun simctl create custom-test-device "$DEVICE_TYPE" "com.apple.CoreSimulator.SimRuntime.iOS-${ios_version_dash}"
+        xcrun simctl list devices "$IOS_VERSION"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/actions/setup-ios-runtime/action.yml around lines 34 - 39, Update
the “Create custom iOS simulator” step to pass inputs.version and inputs.device
through the step’s environment, then reference those environment variables
within the bash script instead of directly interpolating GitHub expressions.
Preserve the existing version transformation and xcrun simctl commands.
sample_app/fastlane/Fastfile (2)

39-43: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

No readiness check after starting the mock server in the background.

bundle exec ruby driver.rb ... & returns as soon as the shell backgrounds the process; there's no polling/health-check before run_e2e_test proceeds to boot the app and run tests. This can cause flaky failures on the very first test if the mock server hasn't finished initializing yet (partially masked by sh_with_retries, but that adds latency/flakiness rather than fixing the race).

Consider polling the driver's HTTP endpoint (similar to stop_mock_server's /stop call) until it responds before returning from start_mock_server.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sample_app/fastlane/Fastfile` around lines 39 - 43, Update start_mock_server
to poll the mock server’s HTTP endpoint after launching driver.rb in the
background, reusing the endpoint pattern from stop_mock_server’s /stop call.
Wait until the endpoint responds successfully before returning, while preserving
the existing startup command and log setup.

12-14: 📐 Maintainability & Code Quality | 🔵 Trivial

Fragile regex-chain path derivation.

root_path strips fastlane/android/ios/// via sequential greedy regex .subs based on string content rather than actual directory structure. It appears to work for the known call sites (sample_app/android, sample_app/ios, sample_app), but is brittle to unrelated path segments containing these substrings.

Consider deriving this from a known relative offset (e.g. File.expand_path('../..', __dir__) from the Fastfile's own location) instead of string substitution.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sample_app/fastlane/Fastfile` around lines 12 - 14, Update root_path in the
Fastfile to derive the project root from the Fastfile’s directory using
filesystem-aware path resolution and the known relative directory depth, rather
than chaining content-based regex substitutions. Preserve the existing root path
returned for the sample_app Android, iOS, and base call sites.
.github/workflows/e2e_test.yml (1)

28-28: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Checkout steps across both E2E workflows persist Git credentials unnecessarily. None of these jobs push back to the repo, so disabling credential persistence reduces the impact if a later step in the job is compromised (flagged by zizmor as artipacked at all four sites).

  • .github/workflows/e2e_test.yml#L28-L28: add persist-credentials: false to the android job's actions/checkout@v6 step.
  • .github/workflows/e2e_test.yml#L80-L80: add persist-credentials: false to the ios job's actions/checkout@v6 step.
  • .github/workflows/e2e_test_cron.yml#L34-L34: add persist-credentials: false to the android job's actions/checkout@v6 step.
  • .github/workflows/e2e_test_cron.yml#L98-L98: add persist-credentials: false to the ios job's actions/checkout@v6 step.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/e2e_test.yml at line 28, Disable Git credential
persistence on every checkout step: add persist-credentials: false to the
android and ios checkout steps in .github/workflows/e2e_test.yml (lines 28 and
80) and .github/workflows/e2e_test_cron.yml (lines 34 and 98), alongside each
actions/checkout@v6 usage.

Source: Linters/SAST tools

.github/workflows/e2e_test_cron.yml (1)

27-163: 🩺 Stability & Availability | 🔵 Trivial

No job-level timeout-minutes for android/ios jobs.

Individual steps have timeouts (timeout-minutes: 90 / 60), but other steps (Bootstrap, Allure launch/upload) are unbounded, so a hang there would tie up a runner up to GitHub's default 6-hour job cap across the whole matrix.

Consider adding a job-level timeout-minutes as a backstop for the nightly matrix.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/e2e_test_cron.yml around lines 27 - 163, add job-level
timeout-minutes limits to both the android and ios jobs as a backstop for
unbounded steps such as Bootstrap and Allure operations. Choose durations that
accommodate the existing emulator and runtime setup time while preventing a hung
matrix job from reaching GitHub’s default six-hour cap; keep the existing
step-level timeouts unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/distribute_internal.yml:
- Around line 50-53: Update the manual workflow dispatch step around the
platform assignment to avoid directly interpolating github.event.inputs.platform
into the shell script. Pass the workflow input through the step’s environment
configuration, then read the environment variable inside bash before the
existing “Selected platform” output, preserving the current platform-selection
behavior.

In @.github/workflows/e2e_test_cron.yml:
- Line 62: Update the scripts invoking fastlane around the affected e2e test
steps to pass mock_server_branch through the step environment, then reference it
as a quoted shell variable rather than interpolating the workflow expression
directly into command text. Apply the same change to both occurrences,
preserving the existing fastlane arguments and secret handling.
- Around line 51-52: Reapply the shared-launch workflow around the Android and
iOS matrix jobs: add a setup job that runs allure_launch once and exposes its
launch_id output, then make each matrix job depend on setup and pass
needs.setup.outputs.launch_id to its Allure upload and removal steps. Remove the
per-cell allure_launch invocations while preserving the existing matrix
execution and cleanup behavior.
- Around line 149-162: Update the Slack job’s action-slack configuration to
report a failure rather than a cancelled status, and set job_name to identify
the failed Android or iOS dependency instead of the current slack job. Preserve
the existing schedule-only failure condition and notification fields.

In `@sample_app/fastlane/Allurefile`:
- Around line 44-56: Update the upload command in the allure_upload lane to use
the computed results variable, preserving the options[:results] override
established by the existence check instead of hardcoding allure_results_path.
- Around line 31-42: Update the install_allurectl lane to use the existing
allure_version when constructing the GitHub download URL, replacing the
latest-release endpoint with the versioned releases/download path while
preserving the current OS and architecture filename logic.

In `@sample_app/integration_test/support/stream_test_case.dart`:
- Around line 23-26: Update the success path in the test wrapper around
body(tester) to check the Flutter test binding for any pending framework-caught
exception before calling Allure.instance.stopTest with AllureStatus.passed. If
an exception is pending, report the test as failed and preserve the framework
failure; only mark it passed when no pending exception exists.
- Around line 44-48: Ensure the test callback always cleans up StreamTestEnv
resources when setup fails: update the body callback around StreamTestEnv.setUp,
addTearDown, and body(env) to use try-finally and invoke env.tearDown in the
failure path, while preserving normal test teardown behavior without double
cleanup.

In `@sample_app/ios/fastlane/Fastfile`:
- Around line 2-12: Update the xcode_version fallback in the Fastfile from the
invalid '26.2' value to the intended supported Xcode version, using '16.2' if
that is the project’s required CI version. Keep the XCODE_VERSION environment
override and the existing select_xcode flow unchanged.

---

Nitpick comments:
In @.github/actions/setup-ios-runtime/action.yml:
- Around line 34-39: Update the “Create custom iOS simulator” step to pass
inputs.version and inputs.device through the step’s environment, then reference
those environment variables within the bash script instead of directly
interpolating GitHub expressions. Preserve the existing version transformation
and xcrun simctl commands.

In @.github/workflows/e2e_test_cron.yml:
- Around line 27-163: add job-level timeout-minutes limits to both the android
and ios jobs as a backstop for unbounded steps such as Bootstrap and Allure
operations. Choose durations that accommodate the existing emulator and runtime
setup time while preventing a hung matrix job from reaching GitHub’s default
six-hour cap; keep the existing step-level timeouts unchanged.

In @.github/workflows/e2e_test.yml:
- Line 28: Disable Git credential persistence on every checkout step: add
persist-credentials: false to the android and ios checkout steps in
.github/workflows/e2e_test.yml (lines 28 and 80) and
.github/workflows/e2e_test_cron.yml (lines 34 and 98), alongside each
actions/checkout@v6 usage.

In `@sample_app/fastlane/Fastfile`:
- Around line 39-43: Update start_mock_server to poll the mock server’s HTTP
endpoint after launching driver.rb in the background, reusing the endpoint
pattern from stop_mock_server’s /stop call. Wait until the endpoint responds
successfully before returning, while preserving the existing startup command and
log setup.
- Around line 12-14: Update root_path in the Fastfile to derive the project root
from the Fastfile’s directory using filesystem-aware path resolution and the
known relative directory depth, rather than chaining content-based regex
substitutions. Preserve the existing root path returned for the sample_app
Android, iOS, and base call sites.

In `@sample_app/integration_test/support/widget_test_extensions.dart`:
- Around line 14-18: Update tapText to accept an optional index parameter,
resolve the text finder to the indexed match when provided, and pass that
single-widget finder to tap so duplicate text matches do not cause a StateError;
preserve the existing behavior when no index is supplied.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9a5873fb-e3da-41aa-a794-32256c4b1519

📥 Commits

Reviewing files that changed from the base of the PR and between dc20586 and 406fcad.

⛔ Files ignored due to path filters (1)
  • sample_app/Gemfile.lock is excluded by !**/*.lock
📒 Files selected for processing (53)
  • .github/actions/cache-cocoapods/action.yml
  • .github/actions/cache-gradle/action.yml
  • .github/actions/cache-xcode-cas/action.yml
  • .github/actions/enable-kvm/action.yml
  • .github/actions/package_analysis/action.yml
  • .github/actions/pana/action.yml
  • .github/actions/setup-flutter/action.yml
  • .github/actions/setup-ios-runtime/action.yml
  • .github/actions/setup-java/action.yml
  • .github/actions/setup-ruby/action.yml
  • .github/workflows/distribute_external.yml
  • .github/workflows/distribute_internal.yml
  • .github/workflows/e2e_test.yml
  • .github/workflows/e2e_test_cron.yml
  • .github/workflows/stream_flutter_workflow.yml
  • .gitignore
  • melos.yaml
  • sample_app/.ruby-version
  • sample_app/Fastfile
  • sample_app/Gemfile
  • sample_app/android/Gemfile
  • sample_app/android/app/build.gradle
  • sample_app/android/build.gradle
  • sample_app/android/fastlane/Appfile
  • sample_app/android/fastlane/Fastfile
  • sample_app/fastlane/Allurefile
  • sample_app/fastlane/Fastfile
  • sample_app/fastlane/Pluginfile
  • sample_app/integration_test/allure/allure.dart
  • sample_app/integration_test/message_list_test.dart
  • sample_app/integration_test/mock_server/data_types.dart
  • sample_app/integration_test/mock_server/mock_server.dart
  • sample_app/integration_test/pages/channel_list_page.dart
  • sample_app/integration_test/pages/message_list_page.dart
  • sample_app/integration_test/robots/backend_robot.dart
  • sample_app/integration_test/robots/participant_robot.dart
  • sample_app/integration_test/robots/user_robot.dart
  • sample_app/integration_test/robots/user_robot_message_list_asserts.dart
  • sample_app/integration_test/support/predefined_users.dart
  • sample_app/integration_test/support/step.dart
  • sample_app/integration_test/support/stream_test_case.dart
  • sample_app/integration_test/support/stream_test_env.dart
  • sample_app/integration_test/support/widget_test_extensions.dart
  • sample_app/ios/.ruby-version
  • sample_app/ios/Gemfile
  • sample_app/ios/Runner.xcodeproj/xcshareddata/xcschemes/Runner.xcscheme
  • sample_app/ios/Runner/Info.plist
  • sample_app/ios/fastlane/Fastfile
  • sample_app/ios/fastlane/Pluginfile
  • sample_app/lib/app.dart
  • sample_app/lib/auth/auth_controller.dart
  • sample_app/pubspec.yaml
  • sample_app/scripts/install_ios_runtime.sh
💤 Files with no reviewable changes (6)
  • sample_app/android/Gemfile
  • sample_app/ios/.ruby-version
  • sample_app/ios/Gemfile
  • sample_app/ios/fastlane/Pluginfile
  • sample_app/Fastfile
  • sample_app/android/build.gradle
🚧 Files skipped from review as they are similar to previous changes (14)
  • sample_app/integration_test/pages/message_list_page.dart
  • sample_app/integration_test/support/step.dart
  • sample_app/integration_test/robots/user_robot_message_list_asserts.dart
  • sample_app/integration_test/mock_server/data_types.dart
  • sample_app/integration_test/robots/backend_robot.dart
  • sample_app/integration_test/robots/user_robot.dart
  • sample_app/integration_test/support/stream_test_env.dart
  • sample_app/lib/app.dart
  • sample_app/integration_test/pages/channel_list_page.dart
  • .gitignore
  • sample_app/integration_test/mock_server/mock_server.dart
  • sample_app/integration_test/allure/allure.dart
  • sample_app/integration_test/robots/participant_robot.dart
  • sample_app/lib/auth/auth_controller.dart

Comment thread .github/workflows/distribute_internal.yml
Comment thread .github/workflows/e2e_test_cron.yml Outdated
Comment thread .github/workflows/e2e_test_cron.yml
Comment thread .github/workflows/e2e_test_cron.yml
Comment thread sample_app/fastlane/Allurefile
Comment thread sample_app/fastlane/Allurefile
Comment thread sample_app/integration_test/support/stream_test_case.dart
Comment thread sample_app/integration_test/support/stream_test_case.dart
Comment thread sample_app/ios/fastlane/Fastfile
Comment thread sample_app/fastlane/Allurefile Outdated
Comment thread sample_app/fastlane/Fastfile Outdated
Comment thread .github/workflows/e2e_test.yml Outdated

@xsahil03x xsahil03x left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just 2 minor improvements i would make.

  1. Add names to all the steps
  2. Use working-directory in cases where we are doing cd sample_app

- uses: 8398a7/action-slack@v3
with:
status: cancelled
text: "You shall not pass!"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add 1 or 2 user tags here, because there are already quite some messages in the channel every day that I would probably not notice it.
This is an example with Sahil and me: https://github.com/GetStream/stream-chat-flutter/blob/master/.github/workflows/beta_version_analyze.yml#L42

I'm not sure if we can tag flutter support instead.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's probably keep it like this until it's stable, later we could tag flutter support. Otherwise you will get too many tags.

@testableapple
testableapple merged commit 5f7f33c into master Jul 16, 2026
26 checks passed
@testableapple
testableapple deleted the ci/e2e-testing-infra branch July 16, 2026 12:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants